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Abstract 

From the irreducible decompositions' point of view, the structure of the cyclic GL„ (C)-module generated 
by the a-determinant degenerates when a = ±i (1 < < n — 1) (see [8]). In this paper, we show 
that —-^-determinant shares similar properties which the ordinary determinant possesses. From this fact, 
one can define a new (relative) invariant called a wreath determinant. Using (GLm, GI/„ )-duality in the 
sense of Howe, we obtain an expression of a wreath determinant by a certain linear combination of the 
corresponding ordinary minor determinants labeled by suitable rectangular shape tableaux. Also we study 
a wreath determinant analogue of the Vandermonde determinant, and then, investigate symmetric functions 
such as Schur functions in the framework of wreath determinants. Moreover, we examine coefficients which 
we call (n, fc)-sign appeared at the linear expression of the wreath determinant in relation with a zonal 
spherical function of a Young subgroup of the symmetric group &nk- 
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Invariant theory for singular a-determinants 



1 Away from the multiplication law 

There is a notion called the a-determinant for a square matrix in probability theory. It was first introduced 
in [11] and actually appeared as coefficients of the Taylor expansion of dct(/ - aA}-'^/". This expansion has 
applications, in particular, to multivariate binomial and negative binomial distributions. Moreover, recently in 
[9], the a-determinant is use to define a random point process through a study of the Predholm determinants 
of certain integral operators. 

The a-determinant det^°'\x) for a matrix X (see (2.1) for the definition) does not have the multiplication 
property which the ordinary determinant det(X) possesses. It is, however, interesting from a viewpoint of 
invariant theory because the a-dctcrminant is regarded as an interpolation of the determinant (a = —1) and 
permanent (a = 1) — recall that each of them generates an irreducible representation of the general linear group 
GL„(C); as representations of the special linear group 5L„(C), the former defines the trivial representation and 
the latter generates the representation on the space of symmetric n-tensors of (the natural representation on) 
C". These facts raise naturally the following question: 

"Where had the m,ultiplication law gone when a moved away from —1?" 

The multiplication law of the determinant is equivalent to the fact that GL„(C)-det(X) C det(X). Hence, 
it is natural to ask the question what the smallest invariant space containing GL„(C) • det'"\X) is. From this 
point of view, Matsumoto and the second author [8] have studied recently the irreducible decomposition of the 
cyclic module W(0[„) ■ det^"\x) and showed that the structure of the module changes drastically when a is 
contained in the set {±1, ±i, . . . , ±^^}. In fact, one can see that the irreducible decomposition of the cyclic 

module ■ det^"\X) degenerates when a is one of such values. More precisely, if we denote by m^{a) 

the multiplicity of the irreducible highest weight W(g[„)-modulc corresponding to a partition A appeared in 
the decomposition, then, for instance, we have m^(— ^) = when the first component of A is greater than k 
(see (3.1)). Therefore, we shall call a singular if a € {±1, ±|, • • • , ±^^}. This result indicates that if a is 

smg ular, then det("\X) may share some distinguished feature which explains why such a drastic change of the 
module structure happens. The special emphasis in this paper is laid on the study of the case a = {k G 

Z>o). Actually, we first show that det^^'^\X) has a certain alternating property which is considered as a 
generalization of the alternating property of the ordinary determinant (as well as its multilinearity) in Section 

2. We also show that siich an alternating property characterizes the — ^-determinants through the cyclic module 

W(fl[„) ■det^~T'\x) by the effective use of the Young symmetrizer (Section 3). We note that a quantum analogue 
of the a-determinant (which we call quantum a-determinant) is introduced and studied in [6], however, it is 
much more difficult to describe the singular values in the quantum case. 

Under these studies, one of the main purpose of the present paper is to construct an invariant, which we will 
call a wreath determinant, defined by means of a singular a-determinant. In order to obtain this new invariant 
for a rectangular matrix, we consider a. kn x kn matrix gotten from multiplexing a given kn x n matrix A by 
tensoring the 1 x A; matrix (1, 1, . . . , 1). By using the property of a-determinants developed in Section 2 for 
a = — we show that the wreath determinant is a relative invariant for the action of the wreath product 
of symmetric groups &k I &n (see [7]) in Section 4. Furthermore, in Section 5, we give an expression of the 
wreath determinant of kn x n-matrix ^4 by a linear sum of the n-th minor determinants of A labeled by the 
corresponding rec:tangular shaped tableaux. In the derivation of this expression, {GLm, GL„)-duality in the 
sense of [3] provides a guiding principle. We then, beside the expression above, derive anotlic;r expression of 
such a wreath determinant conceptually by the Frobenius reciprocity. As a corollary of the proof, we find that 
the wreath determinant is a relative invariant of (Sfe 1 6„) x Gi„. We also give one remark on the background 
which explains how to get this expression and to understand a structure of the cyclic module Z//(g[„) • det*^"\x)^ 
for a general positive integer £ in the framework of {GL^, GZ/„ )-duality. Note that the latter closely relates a 
problem for calculating a certain plethysm [7, 4] . 

The Cauchy determinant formula (see, e.g. [12]) 
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can be considered as one of the most important determinant formula from the representation theoretic point of 
view. In Section 6, we prove an analogue of the Cauchy determinant formula for the wreath determinants. It 
naturally leads us to study the wreath determinant of a Vandermonde type. The aforementioned study enables 
us to deduce a formula for the Schur functions in terms of the — -l-determinants of the Vandermonde type, 
which is regarded as a —-^-analogue of the expression 

/ N det(.r;^'"""')i<,.j<n 

Sx{xi,..., Xn) = . 

det(a;. •')i<j,j<„ 



The proof is to be done first for the corresponding expressions for the monomial symmetric functions m\{x), 
and then, it can be completed immediately by the well-known linear expression of the Schur function by mx {x) 
using the Kostka numbers (Section 6). 

We further try to understand the coefficients which we call (n, k)-sign appeared at the aforementioned linear 
expression of the wreath determinant in relation with a zonal spherical function of a Young subgroup of the 
symmetric group &nk- At this point, we shall provide one conjecture about a positive definiteness of a certain 
symmetric matrix formed by the spherical function (see Conjecture 7.8). We don't treat the remaining singular 
case a = -I (fc G Z>o). Note that, however, one can deduce the fact m^i^) = from the result in [8], 

where A' denotes the transposition of the partition A as a Young diagram. 

We give an a-analogue of the Laplace expansion formula for a-determinants in Appendix A. 



Conventions 

As usual, N is the set of positive integers and C is the complex number field. For n S N, we denote by ©„ the 
symmetric group of degree n. The cycle number of an element a € &n is written by ^'„(cr). Since the conjugacy 
classes of 6„ are parametrized by the cycle type, i^n is a class function on 6„. In particular, we notice that 
t^n{cr~^) = i^n{<7) for any a G S„ because a and are always S„-conjugate. 

We denote by Matm.„ the set of m x n matrices whose entries belong to a certain commutative C-algebra, and 
we put Mat„ = Mat„,„. We also denote by In = {Sij)i<i,j<n the identity matrix of size n and 1„ = (l)i<ij<n 
the all-one matrix of size n. For a permutation a G &n, -P(o') = (^j<T(j))i<j,j<n is the permutation matrix for a. 

The (complex) general linear group GL„(C) is the group consisting of invertible matrices in Mat„(C). We 
exclusively deal with the complex vector spaces so that we often omit the symbol C and simply write GL^ in 
stead of writing GIy„(C). 

Let us put [N] := {1,2,..., N} for N €N. For a given partition (or Young diagram) A of size N, we denote 
by SSTabjv(A) the set of all semistandard tableaux with shape A whose entries are in [N], and we also denote by 
STab(A) the set of all standard tableaux with shape A. For a semistandard tableau T G SSTabAr(A), we associate 
a sequence wt(T) := {iJ,i,H2, ■ ■ ■ , Mw) of nonnegative integers where /i^ = \{tij = k}\ is the number of entries in 
T which is equal to k. We call wt(r) the weight of T. Notice that a semistandard tableau T G SSTabjv(A) is 
standard if and only if wt(T) = (1,1,..., 1). For a given partition X, n \- N of the same size N, we denote by 
the number of semistandard tableaux T with shape A such that wt(T) = /x. Namely, 

Kxi, = \{T& SSTabjv(A) | wt(r) = /x} | . 

We call Kxfi the Kostka nurnher. We also put = |STab(A)| = Kx^^i i), and denote by ^(A) the depth of the 
diagram A. See [1, 7] for detailed information on partitions and tableaux. 

The irreducible polynomial representations of GLm are highest weight modules and the highest weights are 
identified with partitions such that £(A) < m. We denote by the irreducible GL^-module corresponding 
to the partition A. The irreducible representations of 6„ are also parametrized by partitions of n. We denote 
by the irreducible 6„-module corresponding to the partition A h n. See [12] (or [1]) for detailed information 
on representation theory of GL^n and S„. 
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2 Basic properties of general a-determinants 

Let a be a complex parameter. The a-determinant det^"' A of a square matrix A = (ay)i<ij<„ e Mat„ is 
defined by 

(2.1) det(«U:= ^ a"-'^''("')a^(i)i---a^(„)„. 

ween 

We note that det^"^(*A) = det^"\A) because i'n{w~^) = Vn{w) for any uj e 6„. We also notice that det^"^ is 
multilinear with respect to the column (and/or row) vectors. We mainly deal with the —^-determinants for 
fc G N below, so it is convenient to put 

detfeA= \A\^ := det^-i/'^U. 

We note that deti = dct*-^^-* is the ordinary determinant. 

The a-determinant of the all-one matrix 1„ (i.e. every element equals 1) is calculated as 

(2.2) det(")l„= ^ -Q ^i + ia). 

w^&n l<z<n 

We note that this is the generating function of the Stirling numbers of the first kind (see, e.g. [10]). The 
following lemma is the (shifted) partial sum generalization of the identity above. 

Lemma 2.1. For a subset I of [n] ={1,2,..., n}, put 

S„(/) := |w G S„ X ^ I w{x) = xj. 
Then, for any g G (3„, there exists a nonnegative integer m{g,I) such that 

«jee„(/) i<i<k 

where k = The integer m{g,I) is given by n — Vnigwo) where wq G &n{I) is the unique element such that 
fnigwo) > fnigw) for any w G 6„(/). 

Proof. Take an element h G 6„ such that h ■ I = [k]. We identify &k and &n{[k]) naturally. Since w G &n{I) 
if and only if hwh^^ G 6k, it follows that 



wG&„(i) we&k we&k weSfe 

where g' = hgwQh~^. By the definition of wq and g' , it is easy to see that 
(2.3) Pn{g') > i^nig'w) {W G 6k). 

Assume that g' contains a cycle of the form (j2,«2, Ji,«i) («i,«2 G {1,2, ii ^ Z2 and Ji,j2 stand for 
certain disjoint strings in {1, 2, . . . , n} which are possibly empty). Then it follows that Vnig' ■{h, h)) = i^n{g') + ^ 
because 

{32M,3l,h) ■ («2,«l) = (j2>«2) • (il,«l)- 

This contradicts the inequality (2.3). Therefore, each cycle in the cycle decomposition of g' contains at most 
one element in {1,2,..., A;}. Namely, g' is of the form 

g' = {3k,k) (j2,2)-(ji,l)-/i 

for certain (possibly empty) disjoint strings Ji, • • . , Jfe in {k + 1,. . . ,n} and h G &n{{k -|- 1, . . . ,n}). 
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For distinct elements ii, ... ,ii € {1,2, ... , k}, we have 

(ii2'«2) • ■ iii,...,i2,ii) = {ji,,ii,---,ji^,i2,ji,,ii)- 

This imphes that I distinct cycles in g' turn into one cycle in g' ■ {ii, ... ,i2,ii), that is, 

i^nig') - Vnia' ■ {ii, . . . = i-\. 

Hence, if w e Sfe is of the type V^T"^ • • • fc'"'' , then we have 

fe 



Vn{a') - Vnia'w) = ^n{i -i) = k- i^kiw). 



1=1 

Therefore it follows that 

t«eSfe loeSfc i<i<fe 

This completes the proof. □ 

Let us define the left action of (resp. the right action of 6„) on the set Matm,,„ as permutations of row 
(resp. column) vectors: 

• (aij)i<i<m := (acr-i(i)j)l<i<m (ct G Sm)) 

l<j<n 

(aij)l<i<m • T := (aiT(j))l<j<m (t € 6„). 

Notice that a ■ A = P[u)A and A-t = AP{t) for cr e 6^, t € and A e Mat^.n- If m = n, then we have 



det("\«;-^) = det("K-i(i),)= ^ a"""^"'^) [] "-"^sC^ 



= ^ a"-^''^""^"'") n«sW-W = det("^(a,^(,)) = det^^^A • «;) 



for any w S and any A = (ajj) G Mat„. 
Lemma 2.2. T/ie equality 

n 

wee„{i) i<i<k gee„ i=i 

holds for A = {aij)i<ij<n € Mat„ and I C [n] such that \I\ = k. 
Proof. Using Lemma 2.1, we have 

n n 

{In n 
wee„{i) J i=i i<i<fe se©n i=i 

as we desired. □ 
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As a corollary, we have the following lemma. 
Lemma 2.3. For I C [n] such that |/ 1 > A; and A e Mat„, the equalities 

detk{A-w)= detk{w-A)=0 
«)£©„(/) ween{i) 

hold. In particular, if k + 1 column {row) vectors in A are equal, then detfc^ = 0. □ 

Lemma 2.3 and the multihnearity of det^ yield immediately the 

Lemma 2.4. Let A = (ai, . . . ,a„) G Mat„. If = ■ ■ ■ = = b for some I < ii < • ■ • < ik < n, then 

det/j(ai, . . . ,aj + b, . . . ,a„) = detk{ai, .. .,aj,.. . ,a„) 

for any j G [n] \ {ii, ik}. □ 

When we regard a G S„ as an element in &n+m {m € N) in natural way, we notice that Vn+m{o) = Vn{o)+m. 
Further, if wc take a permutation r G &m and regard t as an element in 6„+„i which leave each letter in [n] 
invariant, then Vn+ra{oT) = Vnio) + v-miT)- This fact readily implies the following simple consequence which 
will be used in the proof of Lemma 4.6 (see also Appendix). 

Lemma 2.5. The equality 

det(«)(^^i t^) =det("\An)det(")(A22) 
holds. In paHicular, det^"\Aii A22) = det^"\Aii) det^"\^22). 

Proof. Suppose that A = (oij) G Mat„+m and An = {aij)i<ij<n, A22 = {aij)n+i<i,j<n+m- We also assume 
that aij=Oifn+l<i<n + m and I < j < n. Then it follows that 

det^^U = Yl a"+'"-'^"+'"("' n 



o-G6„+™([n]) 1=1 i=l 

TGS„+m(n+[rri]) 

n m 

= J2 n ""Mi) n = det^^^^n) de&\A22). 

This proves the claim. □ 



3 Characterization of — f -determinants 

k 

In Lemma 2.3, we prove that det^ has an alternating property among k + 1 column (and/or row) vectors. In 
this section, we show, conversely, this property essentially characterizes det^. 

We denote by 'P(Mat„(C)) the commutative C- algebra consisting of polynomial functions on Mat„(C). The 
Lie algebra of GL„ is denoted by and its universal enveloping algebra is denoted by Z//(flI„). The algebra 
■p(Mat„(C)) has a U{gl„) x Sn-module structure by defining 

{Eij■f){X)=Y,^^k^{X) (l<«,J<n), {a-f){X) = f{X-a) (a G S„) 
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for / G 'P(Mat„(C)) where Eij arc the standard basis of 0[„ and Xij are the standard coordinate functions on 
Mat„(C). We note that this action of Z^(flt„) is obtained as the differential representation of GLn given by 
{g ■ f){X) = fCgX) for g e GL„, which is the contragradient representation of the left regular representation 
on 'P(Mat„(C)). Here *g denotes the transposed matrix of g. 

Let ML„ be a subspace of P(Mat„(C)) consisting of functions which are multilinear with respect to column 
vectors. Clearly, we have 

ML„ = ^ C ■ Xi^i- ■ -Xi^n- 

l<ii ,....in<n 

The subspace ML„ is a x 6„-submodule of P(Mat„(C)). For each fc e N, we put 

AL^ := I / e ML„ 



7c[n], |7|>fc^ /(^■r) = ol 

Tes„(/) J 



where X = {xij)i<ij<n- This subspace AL^ is also W(0l„)-invariant because the actions of and S„ on 

'P(Mat„(C)) commutes each other. We also see that AL^ is S„-invariant since 



^ {a.f){X.r)= Yl fiX-Ta)= ^ /(F • r) 

Tee„(/) res„(/) reSn(CT-i/) 



= 

Y=X-a 



for any / C [n], |/| > fc if / e AL^ and a e S„. Since detfe G AL^ by Lemma 2.3, it follows that AL^ D 
Zi(0lJ-dct,(X). 

Theorem 3.1. The equality AL^ = • detfc(X) /loWs for k = 1,2, ... ,n - 1. 

Proof. In [8] , it is shown that 

(3.1) ZY(f,y-detfe(X)- 0(A^^)®/\ 

Ahn 
\i<k 

where Ai^ denotes the highest weight ZY(0[„)-module of highest weight A, which is the differential representation 
of A^^ and we use the same symbol to indicate it. The irreducible module A4^ is realized in ZY(g[„) • detfc(X) 
as an image of the Young symmetrizer 

CT= Yl sgn(g)gp G C[S„] (TGSTab(A)). 

9ec(T) 

Here C(T) and R{T) are the column group and row group of T respectively (see, e.g. [12]). Hence, to prove 
the opposite inclusion AL^ CU{gl„) ■ detk{X), it is enough to show that each element / in AL^ is killed by the 
Young symmetrizer ct when T G STab(A) and Ai > k. We now prove this. The image ct ■ f of f € AL^ by ct 
is calculated as 

{cT-f){x)= Y sgn(9) Y fix-m-\)= Y sg^w E /(^-p?)- 

For each q G C{T), we see that 

Y f{x-pq)= Y I E ip'i-f)ix-p)\=o 

p&R(qT) p'&R'iiqT) [peRi{qT) } 

since p'q ■ f G AL^ by 6„-invariance of AL^. Here Ri{qT) is the subgroup of R{qT) consisting of permutations 
which moves only the entries in the first row of qT, and R'^ (qT) is the subgroup of R{qT) which leave the first 
row of qT invariant so that R{qT) = Ri{qT) x R[{qT). This completes the proof. □ 
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4 Determinants from a variation on wreath product groups 

Let m,n,k G N. For a matrix A = (ai, . . . , a„) € Mat™ „, we define the column fc-plexing A^'^^ e Mat^ fe„ of A 
by 

k k 

^['=1 := (ai, . . . ,ai, . . . ,a„, . . . ,a„). 

This is nothing but the Kroncckcr product matrix A®{1, . . . ,1) oi A and (1, . . . , 1) € Mati_fc. The row fc-plexing 
j4[^] G Matfem^n of ^ is also defined in a similar way. 





f ai 


bA 


-\ 










63/ 


/ai 


ai 


&i 








Us 




63 



'ai ai ai bi bi bi 
02 02 02 62 &2 &2 
,03 as 03 63 63 63, 



We notice that 

A^^^=A-{Inf\ A[k] = {Im\„yA 

for A G Matm,„. Hence one has the 

Lemma 4.2. Let A e Matm,„. T/ien the equalities 

/lo/rf for P e Matm, Q G Mat„. /n particular, we have 

a-AW = (a-A)W, A[,j.r=(A.T)[,j 
/or G 6„, r G S„. □ 
Definition 4.3. For a rectangular matrix A = {aij)i<i<kn G Matfe„,„, we define the A;-th wreath determinant 

l<j<n 

of A by 

fcn-i/fc„(CT) n A; 

nnM(p-i)fc+o,p- 

By Lemma 2.4, it is immediate to see that the equalities 

wrdetfe(ai, . . . , a,_i, + caj,a,+i, . . . , a„) = wrdetfc(ai, . . . , fli-i, a^, a^+i, . . . , a„) (17^ j), 
wrdetfe(ai, . . . , ca^, . . . , a„) = c'" wrdetfe(ai, . . . , ai, . . . ,a„) 

hold for A = (ai, . . . , a„) G Matfe„_„ and c G C. Then it also follows that 

(4.1) wrdetfe(A • a) = (sgncr)'' wrdet^ A (cr G S„) . 

In general, we have the 

Lemma 4.4. If A £ Matkn,n and P G Mat„, then 

wrdetfe(AP) = (det P)*' wrdetfc(A). 

Namely, wrdetfe is a relative invariant ofGLn in 7'(Matfe„,„(C)) with respect to the {right) regular representation 
{See also Section 5). □ 
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Example 4.5. Lemma 4.4 says that the equaUty 

(4.2) det("\(^P)['=l) = (det det("\^['=l) 

holds when a = —1/k. When k — 1 and a — —1, this is nothing but the multiplicativity of the ordinary 
determinant. We also notice that (4.2) becomes trivial when a = —1, —1/2, . . . , — l/(fc — 1)- Actually, because 
of Lemma 2.3, each side of (4.2) vanishes for such values. Further, wc notice that (4.2) holds only if a = 
-l,-l/2,...,-l/fc. Actually, if det^^^xW) satisfies (4.2), then the ratio det^^^X^)/ wrdetfc(A:) gives an 
absolute invariant of GLn, which must be a constant. If the constant is 0, then it follows from (2.2) that 
a = —1, —1/2, . . . , — l/(fc — 1). If the constant is not 0, then we immediately have a = —1/k. Here we give a 
simple and direct example. When n = k = 2 and P = ( J } ), we have 

dct("\(^P)[^l) - (detP)2det("\y4l2l) 
=(1 + a)(l + 2a) ((1 + 3a)aiia2ia3ia4i + 2a(ai2a2i + 011022)031041 + (1+ 0)011021(032041 + 031O42)) 

which is identically zero only if a = — 1, — |. See also Corollary 5.8. ■ 
Lemma 4.6. If A £ Mat„, then the equality 

(J I \ n 

holds for any fc e N. 

Proof. By Lemmas 4.2 and 4.4, we have 

detfc = wrdetfc {A[k]) = wrdetfe ({In)[k] ' ^) 

= wrdetfe ((/„)[fe]) ■ (det A)^ = detfe ((/„)[fe|) • (det^)^ 

n 

Since (/„)|^j = Ife • • • Ife and detfe(lfe) = ni<i<fe(l - i) = ^^^ve detfe ((/n){fej) = (|^)" by Lemma 

2.5. This completes the proof. □ 

This Lemma will be used in §7. 

We consider the two injective homomorphisms (j) : 6^ &kn and ip : S„ Sfe„ defined as 

(l){ai,...,an) : [kn] 9 {i - l)k + j 1 — > {i - l)k + ai{j) e [kn] {1 < i < n, 1 < j < k), 
tp{T) : [kn] 9 (i - l)k + j\ — > (T(i) - l)k +jG [kn] {1 < i < n, 1 < j < k) 

for (cti, . . . , (T„) e &^ and r € &n- To avoid the confusion, we put 5^ := 0(6)!) and S'„ := ijj{&n)- We note 

that 5^ is the Young subgroup 6(fen) of 6fe„ corresponding to the partition (fc") h fen. 

By the definition of /c-plexing, one finds that A^'^^ ■ a = A^'^^ for A e Matfe„,„ and a € SJ^, whence it follows 
that 

wrdetfe {a-A)= detfe (a ■ AI'^I) = detfe (vll'^l • ct) = detfe (^l*^!) = wrdetfe A (a e 5^) . 

We also see thatyl['=l-V(T) = (A-r)['=l for any r € S„. Hence we have 

wrdetfe (V'(r) • ^) = detfe (v(t) • A^) = detfe • V(t)) 

= wrdetfe {A-t) = (sgn r)'' wrdetfe A (re S„) 

by (4.1). Consequently, we obtain the 
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Lemma 4.7. If A G Mathn,n, then 

wrdetfc {g-A) = XnMd)'' wrdetfc A 

for any g £ &kl &n- In other words, C • wrdetfc C P(Matfc„,„) defines a one- dimensional representation of 
&h I &n ■ Here &k I &n ■= SJ^ >^ Sn is the wreath product group {see [7] ) . The character Xn,k of &k I &n is defined 
by 

Xn,k{g) = sgnr 

for g= ((/>(cTi, . . . ,a„); V'(t)) (cTj G Sfc, r e 6„). □ 

5 Expressions of wreath determinants and (GLkn, CL^j-duality 

For given two linear spaces V and W, as a GL{V) x GZ/( W^)-niodule, the multiplicity-free decomposition 
(5.1) S{V^W)^^M^MM^ 

A 

of the symmetric algebra S{V®W) holds. Here A runs over the partitions such that ^(A) < min{dim V,dim W}. 
This fact is referred as {GL{V),GL{W))-duahty (see [3] and [12]). 

The algebra 'P{Matkn,n) has a GLkn x GL„-module structure given by 

((<?, h).f){A) := fCgAh) {g £ GLkn, h £ Gi„, A e Matfc„,„), 

where *g denotes the transposition of g with respect to the standard coordinate. We see that 

P(Matfc„,„) ^ r ((C^")* (g) (C")*) ^ S (C'^" (8> C") 

as GLkn X GX„-module. Here V* indicates the contragradient representation of V. We notice that if {p, V) is 
a representation of GL^, then p{g) = p{*g~^) {g G GL^) defines a representation on V which is equivalent to 
V*. 

Remark 5.1. It is standard to define a representation of GLkn x GLn on the algebra 7-'(Matfc„,„(C)) by 

{{g, h).f){A) := fig-'^Ah) {g e GLkn, h G GLn, A G Matfe„,„), 

which is a combination of the left regular action of GLkn and the right regular action of GLn- If we adopt 
this one, however, then it is no longer a polynomial representation. Instead, in our argument, we adopt the 
contragradient of the left regular action of GLkn so that each (irreducible) factor of the GLkn x GL„-module 
■p(Matfe„,„(C)) is polynomial. ■ 

By {GLkn, GL„)-duality, one has the multiplicity-free decomposition of ■p(Matfe„,„): 

7'(Matfc„,„)- Ml^^Ml- 

l{X)<n 

If we look at the det-eigenspace with respect to the left action of the diagonal torus Tkn — (C^)*^" of GLkn, 
then we have 

P(Matfc„,„)^->^- - {MLf'-'^^'mMi 

l(\)<n 

Here, for a GXfc„-module V, we denote by y^fc^.det ^j^g det-eigenspace 

^T,„,det = |„ g 1/ i y = det{t)v {t G Tkn)] 



K. Kimoto and M. Wakayama. 
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with respect to Tkn- Since the symmetric group &kn is the normalizer of Tkn in GLkn, each det-eigenspace 
(A1^„)"^'""''*'* becomes a S;j„-module. It is known that the equivalence {■Mkn)^''"''^^^ — "^kn holds as &kn- 
modules if A is a partition of kn (sec, e.g. [3]). 

Let us denote by M^^k the irreducible GLkn x GL„-submodule of ^(Matfcn,^) corresponding to the partition 
(fc"), that is, Mn^k — ■'^i^ ^ 1^ -Mn As 6fe„,-modules, we have the equivalence 



since the multiplicity space A4n ^ is of dimension one. In particular, we have dimMj"^"'"^^* = 
By Lemma 4.4 and (GLfe„, Gi„)-duality, it follows that wrdet/j G Mn,k- Moreover, since 

(fen \ 
Y[ Ci j wrdetfe A, 

it follows that wrdet^ belongs to M^^^''^^\ 

For each standard tableau T = (tij)i<i<„ G STab((A;")), we define the function dety on Matfe„,„ by 

i<i<fe 

k 

detT(^) := Udet(at4,j)i<,j<„ {A = (ay)i<i<fe„ e Matfe„,„). 

1=1 i^^J^" 

We also define the matrix I{T) G Mati;„^„ so that t^-th row vector of I{T) is equal to the i-th fundamental row 

j-th 

vector ei = (0, . . . , 0, 1 , 0, . . . , 0) for each i = 1, . . . , n and j = 1, . . . ,k. In other words, if we define g{T) € &kn 
for T e STab((A;")) by 

(5.2) g{T){{i-l)k + j) =Uj {l<i<n,l<j <k), 

then I{T) = g{T) ■ {In)[k]- Denote by Tq the standard tableau with shape (fc") whose (i, j)-entry is (i — l)k + j. 
We note that g{T) e &kn is the permutation determined by g{T) • Tq = T for each T G STab((fc")). 

Lemma 5.2. For T,U G STab((fc")), the equality 



detT(/(C/)) 



1 T = U 
T^U 



holds. 



Proof. When T = U, the tu-th row vector I{T)t^i of /(T) is equal to Bi ii i E [n] and I G [k], and hence 
detT(-'^(T')) = 1. When T = (tij) and U = (u^) are distinct standard tableaux of shape (A:"), there exists a pair 
(si, S2) of distinct elements in [kn] such that si and S2 are in the same column of T and in the same row of U, 
say si = fjjc = Urji and S2 = tijc = '"rj2 (h 7^ «2j ji 32)- Then we have 

I{U)t,^ = I{U)u^ = Br, 

which implies that det(/(f7)tj^ j)i<ij<„ = 0, and hence detT{I{U)) = 0. □ 
Theorem 5.3. The wreath determinant wrdetfe A of a matrix A G Matfe„,„ is expressed as a linear combination 

wrdetfe A = ^ wrdetfe /(T) •detT(^) 

TeSTab((fe")) 

o/detT(^) for T G STab((A;"')). The coefficient wrdetfe I{T) is given by the sum 

^ \ kn-Uk„{g{T)a) 

k. 



wrdetfe /(T)= 



where g{T) G &kn is a permutation defined by (5.2). 
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Proof. We observe that dcty (A) is a homogeneous polynomial in Uij of degree kn satisfying the condition that 
detr(AP) = (detP)'= detT(A) for any P € Mat„. We also see that 

/ kn \ 

(diag(ci, . . . ,Cfe„).detT)(A) = detT('diag(ci, . . . , Cfe„)A) = J^Ci detr^. 



Thus, it follows that every detr belongs to AfJ^^T''^'^' by (GLfc™, GL„)-duality. 
We show that {detT}TeSTab((/s")) are linearly independent. Suppose that 

CTdetT{A) = 

TeSTab((/s'')) 

for any A £ Mat/c„,„. Then, by Lemma 5.2, we have 

= C"r detT(/(J7)) = Cu 

TeSTab((/s")) 

for each U G STab((fc")), which assures the linear independence of {detT}TGSTab((/s"))- Since dimMj*^"'*^*^* = 
f^''"\ it follows that {detT}TeSTab((/s'')) is a basis of mJ^^"''^'^*. Hence wrdet/j is written as 

wrdetfeA= ^ C^detT(^) (A e Matfc„,„). 

TeSTab((fe'»)) 

By Lemma 5.2 again, the coefficient for U G STab((A;")) is calculated as 

wrdetfc/([/) = ^CydetT (/([/)) = CidetuiHU)) = C^. 

T 

This completes the proof of the theorem. (The coefficient wrdetfe I{U) is calculated later in Section 7.) □ 
Example 5.4. When n = 3 and fc = 2, there are five standard tableaux with shape (2^): 



1 


2 


3 


4 


5 


6 



U2 



1 


2 


3 


5 


4 


6 



1 


3 


2 


4 


5 


6 



1 


3 




1 


4 


2 


5 




2 


5 


4 


6 




3 


6 



(We remark that To = Ui in this case.) The corresponding matrices I{Up) are given by 



nui) 



(I 









(I 









(I 







1 










1 













1 










1 

1 







, I{U2) = 






1 






1 




1 






1 













1 







1 













1 







y 


















1/ 


/I 





o\ 




/I 





o\ 











1 










1 











1 
















1 














1 


1 

















1 










1 











^0 





V 


















I{U,) 



and their 2-wreath determinants are calculated as 

wrdet2 1{Ui) = wrdet2 1{U2) = wrdet2 ^(f/s) = 



1 

16' 



wrdet2 1{U4) = wrdet2 -^(f/s) = 



32' 



K. Kimoto and M. Wakayama. 
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Thus we have 

wrdet2 A=^ detc/i {A) - ^ det^, {A) - ^ detu^, {A) + ^ dety, (A) + detu, (A) 

for A G Mat6,3. ■ 

As a corollary of the theorem, wc obviously have the 

Corollary 5.5. For A G Matp^„ and B G Matg^„, we denote by ASB G Matp+q,„ the matrix obtained by piling 
A on B. If Ai, . . . ,Ak & Mat„,„, then the equality 



wrdetfc(Ai ffl •• • ffl Afe) = ^ wrdetfe/(T) JJdetBj(T) 

TeSTab((/c'')) 



i=l 



holds, where Bj{T) is a matrix whose i-th row is equal to the tij-th row of AiS ■ ■ ■ S Ak. 
Example 5.6. If A = 



an ai2 

^21 022 



wrdet2 {AS B) = wrdet2 



and B 


bii 


bi2\ 


\b2i 


b22)' 


1 ail 


ai2\ 




1 




021 


a22 




ail 




bi2 




4 


021 


V&21 


b22 J 









^11 bi2 

&21 b22 



ail ai2 
bii bi2 



021 «22 
&21 &22 



Recall that the wreath determinant wrdetfe is S^-invariant. By the Probenius reciprocity, it follows that 



dim(M-;--) 



ST 



res^ 



where {V, W)q denotes the intertwining number of two G-modules V and W, and Iq is the trivial representation 
of G. Hence we have 



(5.3) 



(<f'"^0^' = C-wrdet,(X). 



This fact implies that J2aeSj^ /(^ ' proportional to widetk{X) for any / G m^''/^''^^*^ . Therefore, we have 

detTo(CT • X) = Cwrdetfc(X) 

for a certain constant C. If we set X = {In)[k], then we have 



c 



wrdetfc (/„)[,] aesn 

Consequently, we obtain another (symmetric) expression of wrdetfc(X) as follows. 
Corollary 5.7. The equality 

wrdetfe(A) detTo(fT • A) 



holds for any A G Matfen.n • 

As a corollary of the discussion above, we obtain the 



□ 
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Corollary 5.8 (Characterization of the wreath determinant). Put 

P(Matfc„,„)'^">'='^^*' = {/ e 7'(Matfc„,„) | /(aXP) = Xn,fe(<7)'=(detP)'=/(^), a e 6^ ! 6„, P e 

Then V{M&tkn,n)'^''''"'^'^^ is a one dimensional subspace spanned by wrdetk- Namely, the equality 

P(Matfc„,„)'^n,..det'= ^ c • wrdetfc(X) 

holds. □ 

Corollary 5.8 and Example 4.5 suggest the following problem: Describe the irreducible decomposition and 

singular values of the cyclic module W(0[j,,J • dot^"\x!'^l) C 7'(Matfe„,„) (X = {xij)i<i<kn,i<j<n)- This is solved 
in the following way. If a = 0, then we see that 



UislkJ ■ detW(X['=l) ^ 5'=(C^")®" ^ (Mt) 



Xhkn 



by a similar discussion in [5] (we also refer to [8] for the case where k = 1). By [8], the A-isotypic component 
of the module U{glf,^) ■ det'"\x) C 'P(Matfe„) does have a positive multiphcity if and only if fx{a) ^ and 
is given by U{glkn) ■ IinniA(^) (We put X = {xij)i<ij<kn in order to avoid confusion). Here lmmx{X) is the 
immanant of X for A and /a (a) := j)exi'^ + ~ *)'^) (modified) content polynomial for A. Since the 

map ^(Matfcn) 9 f{X) t— > /(Xl''!) e P(Matfe„,„) defines a GZ/fe„-intertwiner, we see that 

the A-isotypic component of Z^(0[,J • det(")(x[^l) ^ ' ^--^(XW) Ma) + 

1 otherwise 

for A h fcn. Thus it follows that i/(0[fe„) •ImmA(XW) ^ (A^L)*^^'"""'- Hence we obtain the following theorem 
which is regarded as a generalization of the result in [8]. 

Theorem 5.9. The irreducible decomposition of the cyclic module generated by det^"\X^''^) is given by 

W(0l,J.det("\xW)- (AlL)*''""'"'- 

Ahfcn 
/A(a)#0 

In particular, the singular values are given as roots of the content polynomials. □ 
Remcirks on this section 

Let <S(C") = J2k>o '5'^(C") be the homogeneous decomposition of 5(C"). Each symmetric power 5'^(C"), that 
is, the space of A;-th symmetric tensors defines an irreducible G'L„(C)-module [1]. We see that the eigenspace 
decomposition of the GL^ x GL„-module S{C"^ ® C") with respect to the diagonal torus of GLm(C) is 
given by 

5(C'"(^C")^ 5'=i(C")(8)---®<S'='"(C"). 

fei,...,fe„>0 

Hence the m-th tensor product 5'=(C")®™ can be identified to the det'^-eigenspace 

5(C'" O C")^-'^''*' = G 5(C'" ® C") t.v = {dett)''v {t e r„)} 
for Tm [3]. By {GLm, Gin)-duality (5.1), we see that 

(5.4) 5fc^c«^®™ ^ ^^^rn ^ (^n^T^.det-- ^ ^ ^_yy^A^^T„,dct'= ^ j^X^ 

£{X)<iniii{m,n} 



K. Kimoto and M. Wakayama. 
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We notice that (A^^)'^""'^"* = {0} unless A h km, and hence the last sum (5.4) is effectively over the partitions 
of km. Note also that dim(Ai^)"^'"''^'** = ^^^(fem) and (A^^)-^"''^°' is stable under the action of the Weyl group 
&m of GLmi'C). We note that the decomposition (5.4) for fc = 1 gives (S^, GL„)-duality (Schur duality) 



(5.5) (C"-)0m^ 



\\-mJ\\)<n 



Suppose now A h km. The group &kl&m acts on the weight space (Al^)^""'^'^' because the wreath product 
&kl&m = S^y\ Sm is obviously acting on the space 5'=(C")®'". Since 6fc acts on 5'=(C")®'" trivially, its action 
on the weight space (A^^)^""'^''* is also trivial. Hence, (5.4) does not provide the irreducible decomposition as 

a bi-module of {&kl&m, GLn{C)). Then, the question how the space (7\4,^J"^"''^°' decomposes as a 6„i-module 
comes into being. Now we establish this question in a concrete way. From Schur duality, as a &m x GL(iS'^(C"))- 
module, we obtain 

li\-m,i{fj.)<N 

where N = dim.S^{C") = ("^^~^) > n. Decompose the module M.^ of GL(<S'^(C")) into irreducible ones as a 
representation of the subgroup GL„(C) of GL(iS*^(C")): 



GL„{C) ^ ^ ^' 

\J{X)<n 



mx{iJ,) being the multiplicity of Ai^ in the irreducible summands of the restriction. Then we have 



X,i{X)<n /iHm 



Therefore, it follows from (5.4) that 



(5.6) Yl m®"'^^^'^ = i^r 

The procedure explained above is a special case of the problem for computing plethysm (or the functorial 
composition of operations A i— > J^^) (see [7], [4]). Note also that the problem for describing the decomposition 
(5.6) for A h km explicitly comes up naturally when one wants to know the structure of the cyclic GL„(C)- 
module generated by det("\X)'= [k = (1, )2, 3, . . .) (see [5]). 

6 Formulas for wreath determinants a la Cauchy et van der Monde 

We give an analogue of the Cauchy determinant formula in the context of wreath determinants developed in 
the previous sections. 

Proposition 6.1. Let n, fc e N and x\,. . . , Xkn, Vi, ■ ■ ■ ,yn be commutative variables. Put 

C„,fc(a;,y)=( — ]■ — ) , Vn,k{x) = {x"~^)l<i<kn■ 

\Xi + yj J i<i<kn l<j<n 

l<j<n 

Then we have 

(6.1) wrdetfc G„,fe(x, y) = ^"^'^^ wrdet^ Vn,k{x). 

l<i<kn 
l<j<n 
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Here A„(y) denotes the difference product 



l<i<j<n 



Proof. For a rational function f{t) in variable we write 

fix.) ■■= ] 

Using this convention, we have 



e Matfen.i 



C„,fc(a;,2/) = ( — ] ] ), Vn,k{x) = {x'2 \...,a;*,l). 



By Lemma 4.4, wc have 

1 1 



wrdetfc 
= wrdetfc 
= wrdetfc 



111 11 



Xi, + 1/1 + J/2 Xi, + yi a;* + 1/„ + yi 

1 2/1 - 2/2 ?/i - 



x^ + l/i (x* + yi)(.T* + 1/2) (a;* + 2/i)(a;* + 2/„), 

1 1 1 



={yi - 2/2) • • • (yi - yn) wrdet^ 



+ yi ' (a;* + yi)(a;* + y2) ' " ' ' (a;* + yi)(a;* + y„] 
Iterating this procedure, we reach to the expression 

1 1 



wrdetfc 



a;* + yi a;* + y„ 



(1 1 "1 

^ w ^ w — ^^'•••'11 7 — 
a;* + yi (a;* + yi ) (a;* + y2 ) (a;* + yj ) 



Using the multihnearity of detfc with respect to the row vectors, we have 



. / I 1 A 1 

wrdetfc ■ ,- ■ ■ 7,---:ll7 . 

\ a;* + yi (a;* + yi)(a;* + y2) jj-^ {x^ + y^) 

1 / " " \ 

= n — Z — wrdetfcrQ(a;* + yj),JJ(a;* + yj),...,(a;*+y„),l . 

l<j<n 

The last wreath determinant is equal to wrdetfc(a;"~^, . . . , x^,, 1) = wrdetfc Ki,fc(a;) by Lemma 4.4. This completes 
the proof. □ 

We note that the proof above is exactly a wreath-analogue of the one of the Cauchy formula [12]. 
Example 6.2 (fc = 1). When k = 1, formula (6.1) is nothing but the ordinary Cauchy determinant formula 

det/^ ^ ^ _ A„(a;)A„(y) 



Xi + Vj J l<ij<n Yli,j=l{Xi + Vj) 



K. Kimoto and M. Wakayama. 
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Example 6.3 {k = 2). When k = 2, (6.1) gives the formula 



wrdet2 



1 


1 


1 \ 


xi + yi 


xi+y2 


Xl + Un 


X2 +yi 


X2+y2 


X2 + yn 


1 


1 


1 


X2n + yi 


X2n + 2/2 


X2n + yn/ 



n 

l<2<j? <n 

n (^i+vj) 

l<i<2n 
l<j<n 



(xl 



wrdet2 



X2 



\x. 



,n-l 
2n 



Xl l\ 
X2 1 



X2n 1/ 



We notice that the other variant of this Cauchy-type identity also follows immediately from (6.1). Indeed, 
we have 



wrdetfe 



1 



1 



A„(y)* 



1 - Xi,yi ' ■ ■ ■ ' 1 - ni<i<fen(l - Xiyj) 

l<3<n 



wrdetfe Vn,k{x), 



which is a wreath determinant analogue of the formula 

1 \ _ A„(a;)A„(j/) 



det 



, 1 - XiVj J i<i_^.<„ n"j=i(l - Xiyj) ' 
As a corollary of Theorem 5.3, we have the 
Theorem 6.4. The wreath Vandermonde determinant wrdetk Vn,k{x) is given by 

wrdetfe Vn,k{x) = ^ wrdetfe /(T) • At{x), 

TeSTab((fe")) 

where At{x) is the Specht polynomial for a standard tableau T = (tij) e STab((fc")) defined by the product 

k 

At{x) := JjA„(a;t„,...,a;t„J 

1=1 

of difference products. 

Another (symmetric) expression for wrdetfe Vn,k{x) also follows from Corollary 5.7. 
Theorem 6.5. The equality 

wrdetfe Vn,k{x) = -jj^ ^ a • A„,fe(x) 



□ 



holds where A„_fe(x) is given by 

k 

A„,fe(x) :=YlAn{xi,xi+k, ■ ■ ■ ,xi+(^„_i)k) = Ato{x). 



1=1 



□ 



For a partition A = (Ai, . . . , Ajv) of depth at most N, the Schur function s\{xi, . . . ,xn) of N variables is 
defined as the ratio of the Vandermonde-type determinants as 



det 



sx{xi,...,xn) 



i,j<N 
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An arbitrary symmetric function can be written as a linear combination of the Schur functions. We show that 
any symmetric function in kn variables can be written as a linear combination of the ratios of the Vandermonde 
type —^-determinants analogously. 

We recall the Cauchy identity concerning the Schur functions (see, e.g. [7, 12]). 

Lemma 6.6. For m,n gN, the equality 

Yi . . ^ S sx{xi,...,Xm)s\{yi,...,yn) 

l<i<m ^'^-^ ^(A)<min{m,Ti} 
l<j<n 

holds. □ 
By the multilinearity of detfc with respect to column vectors, we have the following expansion formula 

/I 1 

wrdetfc 



Thus we have 



(6.2) 



1 - Xi^yi 1 - Xi^yn 

:detfc I J2 i^^yif'^ E {^*y2r\---, E ^^*y^y 

\iii>0 i2i>0 ifc„>0 

ni,i21,---,»fen>0 



ill,»21,---,»fen>0 

= A„(j/)''wrdetfey„,fe(x) ^ sx{xi, . . . ,Xkn)sx{yi, ■ ■ ■ ,yn)- 

l(\)<n 



Comparing the homogeneous terms in (6.2), we have the 
Lemma 6.7. Put 

Kki^^y) ■■= E • . .yH„+-+i.„ detfe (xl",4-, . . . ,4'=") 

»ll,»21,---iifcn>0 
Hl+---+»fcn=rf+ '°"'2~'' 



Then, the equalities 



and 



f I 1 \ °° 

Ht,kix,y) = ^n{yf^^detkVn,k{x) E «A(a:;)sA(2/) 



^(A)<n 
|A|=d 

hold. □ 

Since the Schur functions of n variables are the irreducible characters of the unitary group U{n), it follows 
from (6.2) that 



1 • • • 1 •^kn ) 



Ul,i21,---i^fcn>0 
»l + -+»fcn = |A|+ '°"'r" 



•••-*1"+-+^'="sa(2/) detfe (4",xt^i,...,4'=") 



^ 1 A„(y)fe ""^^^U wrdetfeV;,fc(a;) 



K. Kimoto and M. Wakayama. 
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where T„ is the n-torus in U (n) and dg is its normaHzcd Haar measure. Thus imphcitly, we find the Schur func- 
tion s\{xi,. . . ,Xkn) can be written as a hnear combination of the ratios detfe {xl^^,x^^^ ,.. . , a;^*"" ) /wrdet/c Vn,k{x) 
of Vandermonde type — -^-determinants. Actually, we have the following expression. 

Proposition 6.8. For a given sequence a = (ai, . . . , Ofc„) G Z>q of non-negative integers, put 

DnAx;a) = detk «Oi<ij<fc„- 

Let us also define ei, Sn,k S Z>q by 

e, = (0,. .. ,0,1,0,. ..,0), ,5„,fe = (^n - 1 - j ej. 

Then, the Schur function s\{x) is written as 

^^^'^^^ wrdet.Kfe(x) ^ ^ ifA.-i3„,.(x;5„,, + ^M.we.). 

We notice that wrdct^ Vn,k{x) ~ Dn,kix, Sn^k)- 

For a partition A = (Ai, . . . , Xkn) whose depth is at most kn, the monomial symmetric function m\{x) is 
defined by 

kn 

o-eSfe„/SA i=i 

Here ©A is the stabilizer of A, that is, ©a = {o" G &kn ; Xcr{i) = Aj, 1 < i < fcn}. The proposition follows from 

the following simple lemma. 

Lemma 6.9. Let X be a partition whose depth is at most kn. Then, the monomial symmetric function m\{x) 
has the following expression 

^ kn 

"^^(^^ = wrdet,K.(x) ^ D^,k{x;Sn,k + Y.>^^ii)ei)- 
Proof. For any a G &km we have 

kn / -. \ kf^ — '^-k-nir') kn . . fcn 



Dn,k{x;dn,k + J2Xa{i)ei) = Yl f-l) n^rw" II- 



Hence it follows that 

fen / 1 \ kn-Vkrvir) kn . i_i , / fen ^ 

E ^.,.(^;'^n..+EA^««0- E (-^) n-:«^^^- E n-:(^f 

CTGSjin i=i T6efc„ ^ ^ i=i \CTeefe„i=i / 

= wrdetfe Vn,k{x) |Sa| mA(a;). 



Therefore we obtain 



A(T(i)ei 



TOA(a;) = — J— -TT — TTT^ E ■Dn,fe(a;;5n,fc + E' 
wrdetfcl4,fc(x)|6A|^^^^ 'V ' ^ 

" wrdet^y ^(x) ^ -Dn,fc(a:;5n,fc+E^-W^i 
wraetfc Vn,k[x) ^^e^^/e^ ^ i=i 

This completes the proof. □ 
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Since the Schur functions are written as a linear combination 
of monomial symmetric functions, Proposition 6.8 follows immediately. 

Corollary 6.10. The power-sum symmetric functions Pd{x), the complete symmetric functions hd{x) and the 
elementary symmetric functions ed{x) are expressed as 

^ kn 

Pd{x) = — , , — ^ Dn,k{x; 6n,k + dei), 
' 1=1 

1 

'^^^^^ = wrdet.K,.(.) ^".'=(-;^".'= + E-^0' 

1 

^'^^^) = wrdet.y„.(x) E D^,,{x;Sn,k + Y,e,,). 



<id<kn J=l 



□ 



7 Generalities on (n, /c)-sign and spherical functions 

For k,n gN, we put 



;„,fc := {/ : [kn] ^ [n] | = A;, Vj G [n]}. 



We notice that !H„4 = 6„. We also notice that &kn acts on 9l„,fe transitively from the right, and S„ acts on 

^n,k from the left. 

For / G 9ln,fe, we define the (n, k)-sign of / by 

sgn„,fe(/) := wrdetfc(5/(j)j)i<i<fc„. 

l<j<n 

We see that 

sgn„,fe(T • /) = sgn(T)'= sgn„ 
for T G Sn- Using this sign for / G and the very definition (4.6) of the wreath determinant we have the 
Lemma 7.1. Let fc,n G N. Then the equality 

wrdetfeA^ ^ sgn„;^(/) JJ aif^e, 

f&^n,k ie[kn] 

holds for any A = (aij) G Matfc„_„. □ 
We define the element Ln,k & 9ln,fe by 

I'nMi''' -'^)k + j)=i (1 < i < n, 1 < j < k). 
The stabilizer of i„ ,fe in 6kn is S'fc • Hence, it follows that 

kn—vkn{w) n k 

^fw{{i-l)k+i),i 



-k) nn^ 

/ \ fcn-!^fc„(ti)) 



fen-i/fe„(g(/)u;) 
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where g{f) G &kn is defined by / = -gif). Therefore, if we regard a standard tableau T = (tij) G STab((fc")) 
as an element of 9in,k by the assignment T : [kn] 9 tij i-^ i G [n], then sgn„^(T) = wrdet;;/(T). Hence, the 
result of Theorem 5.3 can be expressed also as 

wrdetftA= J2 sgn„_fe(T)detT(A). 

TeSTab((fe'»)) 

We consider the injection 

UJ : &n^ {Wl, . . .,Wk) I > ((« - l)fc + j Wj{i)) e 9ln,k, 

and denote its image by ^. By Lemmas 4.6 and 7.1, we have 

(^•^) (fcfc) ^SnH [| ]J ai,„(^)((i_i)fc+j) = ^ sgn„_fc(/)]J]Ja,j((i_i)fc+j) 

for {aij)i<i^j<n G Mat„. Comparing the coefficients in both sides, we obtain the 
Corollary 7.2. For any f G D^n.fe; the equality 



sgn„,fe(/) =sgn(M;) ( ^1 



/lo/rfs /or w e 6^ SMc/i that lo{w) G (/ • SJ^) D ^. T/ie sign sgn{w) does not depend on the choice of w. 

Proof. Fix an element / G !EH„,fc. We notice that the monomial HiLi 11^=1 o,ij((i-i)k+j) in the right-hand side 
of (7.1) depends only on the orbit f ■ SJ^. We also notice that the function sgn„ is constant on each SJ^- 
orbit. Hence the coefficient of the monomial 0"=! 11^=1 '^i,/((i-i)fc+j) in the right-hand side is sgn„ \ f ■ S^\. 
For any w = {wi, . . . , Wk) G such that a;(w) G / • 5^, the sign sgn((i,') = sgn(w;i . . . Wk) gives the same 
value, which can be verified by counting the inversion numbers. It follows that the coefficient of the monomial 

nr=i nj=i aiJ{{^-l)k+J) left-hand side is sgn(w;) | (/ • S^) n m^^^ for any w e {f ■ S^) H ^R^^^. Thus we 

have the desired conclusion. □ 

As a corollary of the discussion above, we obtain the 

Proposition 7.3. (1) Put 

mj{f) = \{l^[k]\f{{t-i)k + l)=j}\. 

Then 



(2) The equality 

kn 

sgn„,fe(/)det(yl)'= = ^ sgn„ ^(/i) J| a/(,)^(,) 

holds for any f G 9l„,fe and A = (ajj)i<i,j<n G Mat„. {When k = 1, this is just the definition of the determi- 
nant.) 

(3) For f G Dln,k, put 

^ n k 

P/(a;ii,...,a;„fe) := — - ^ H II 
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Then 



n.k 



\f-s]:\ 



= the coefficient of Y[ Xij in Pf{xii, . . . , Xnk)- 



l<i<n 
l<j<k 



□ 



It is convenient to express an element / e $H„,fe as an n x matrix whose (i, j)-entry is given by f{{i — l)k+j), 
that is, 

/ /(I) ••• f{k) 

/= ; ■•. : 

V/((n-l)fc + l) ... fink), 

If /i, /2 S ^n,k and /2 = /i -cr for some ct G 5^, then each row vector of /2 is a permutation of the corresponding 

row vector of /i . 

Example 7.4. Let us calculate sgn„ ^{1/4,) = wrdet2/(C/4) for U4 (regarding as an element in 9^3,2) given in 
Example 5.4. In the matrix notation, 



C/4 



1 


3 


2 


5 


4 


6 



1 1-^ 1 2 1-^ 2' 
1 4i-^3 
5 2 61-^3 



'1 2^ 
1 3 

.2 3; 



It follows that 



U4 ■ Si 



'I 2> 
1 3 

.2 3; 



'1 2^ 
1 3 
.3 2; 



'1 2^ 
3 1 

.2 3; 



'I 2> 



^2 1> 



3 1,1 3 



,3 2; 



,2 3; 



^2 1^ 
1 3 

.3 2; 



^2 1^ 
3 1 

.2 3; 



^2 1^ 
3 1 

.3 2; 



and 



Since 



(C/4 • S'l) n IH; 



3,2 



^1 2^ 
3 1 

.2 3> 



'1 2> 
3 1 

.2 3; 



^2 1^ 
1 3 

.3 2; 



a;((2,3),(l,2)) 



and sgn((2, 3), (1, 2)) = 1 (where denotes the transposition of i and j), we get 



We remark that 



and we see that 



wrdet2 I{U4) = ( ^ ) x ^ 



^mii{U4) mi2{Ui) mi3{U4) 

17121(114) m22{U4) m23(f/4) 
\msi{U4) m32{U4) m33(f/4). 



213 



213 



1 

32' 





1 













1 





8 = \U4-Si\ 



ni<ij<smj{U4y. 1!1!0!1!0!1!0!1!1! 
as we counted above. We also note that 

Puiixn, X32) = ■^(a;iia;22 + a;i2a;2i)(a;iia;32 + a;i2a;3i)(a;2ia;32 + a;22a;3i), 



and the coefficient of a;iia;2ia;3ia;i2a;22a;32 of Pu4, is ^ = 7- 

8 4 
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Let us put 

for g e &kn- We note that 'Pn,k{g~^) = Vn,k{g) since ykn{g~^(^) = i'kn{g(^~^)- By Lemma 4.7 and its S^- 
invariance of 1®", it follows that 

^n,k{high2) = Xn,k{hlh2)^^n,k{g) 

for g G Sfe„ and fti,/i2 G 6fc I &n- In particular, is a S'^-biinvariant (or S'^'-zonal spherical) function on 
®/cn- We note that the rightmost side of (7.2) can be considered as an analogue of the integral expression of 
the zonal spherical function of a Riemannian symmetric space due to Harish-Chandra (see, e.g. [2]). 

Lemma 7.5. The Xn /.-spherical function (pn,k relative to the wreath product &k I &n on &kn is expressed as a 
matrix element of the {unitary) representation Mj'jr'^°'(= Jkn) of &kn'- 

_ {g ■ wrdetfc(X), wrdetfc(X)) 
^nM9)- (wrdetfc(X), wrdetfe(X)) ' 

t inner product 

Proof. Consider the projection 



where { , ) denotes the invariant inner product on M^''^''^"^. In particular, (pn,k is a positive definite function. 



By (5.3), for each g G &kn, there exists a constant C{g) such that 

(7.3) Pn,kg- wrdetfc(X) = C{g) wrdetfe(X). 

Since Pn,k is self-adjoint with respect to ( , ) and wrdetfc(X) is S'^-invariant, it follows that 

{g ■ wrdetfe(X), wrdetfc(X)) = {g ■ wrdetfc(X), P„,fc • wrdetfc(X)) 

= {Pn,kg ■ wrdetfe(X), wrdetfc(X)) = C{g) (wrdetfc(X), wrdctfc(X)) . 

To determine C{g), let us calculate the coefficient of 11^=1 nf=i ^(p-i)fc+i,p in the both sides of (7.3). It is 
immediate to see that the coefficient in the right-hand side is C{g) (^)" = C{g)detk (ifc")- We look at the 
left-hand side: 

Pn,k g ■ wrdetfe(X) = ^ ag ■ wrdet;o(X) 



1 / -^\kn—vkn{h) n k 

^Jq^lYl [~k) '[l'[l^icrgh)iip-l)k+l),p 




„ A; 



nn^^((p-i)fe+o.p- 



Hence the coefficient of Y[p=i Ilf^i ^{p-i)k+i.p in Pn.k ,9 ■ wrdetfc(X) is equal to 
Thus we have 



detfe (g-in , , 

det.(ir) 

This completes the proof. □ 
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Remark 7.6. By specializing the Probenius character formula for &n, we have 



\\-N 



{kn)l |SSTabfe(A')| 



where f\{a) denotes the content polynomial defined by 
Since 

h{-l) = n (i-^^--^))=P^ n ik+a-j))- J. 

^ ^ (i,i)eA ^ ^ (i,i)6A' 

it follows that 

^xfcn-..„(s) ^ |SSTabfe(AO| , 

^ Ahfen 

Hence the function ipn,k is a linear combination 

^n,k{9)= |SSTabfe(A')|0^,fe(5) 

Ahfcn 

of S^-zonal spherical functions 



with nonnegative (integral) coefficients. Therefore, it is immediate to see again that Lpn,k is a positive definite 
function. ■ 

Remark 7.7. Since ^indfl" 1^^, J^,^ = Kx^{kn) for A h fcn, the pair (6fe„, 5^) is not a Gelfand pair in general. 
Further, although one can verify that the pair (6fe„, &k I S„) is a Gelfand pair when k = 2 (sec p. 401 in [7], 
in fact, the wreath product 62 I &n is isomorphic to the hyperoctahedral group of degree n), it is not the case 
for a general k. Actually, when n = 3, by looking at the Schur function expansion of the plethysm /13 o hk (see 
p. 141 in [7]), it follows that the induced representation indg^^g^ IsjiiSa is not multiplicity free when fc > 18. ■ 

For a standard tableau T e STab((fc"')), we define 

DT{X)=wvdetk{9{T)-^-X), 
where g{T) is a permutation given in (5.2). We see that 

Dt{X)= wrdetfe(5(T)-i/(5))dets(X) 

SGSTab((fc")) 
7 I \ n 

SeSTab((/c")) 



We now define the x matrix En,k by 

(7-4) = {VnA9iT)-'9iS)))s,T^sTM(k-)) • 

Since ipn,k{9) = ^n,k{9~^), 

one finds that the matrix ^n.k 

is symmetric. Moreover, we notice that det En,k > 
by Lemma 7.5, because ipn,k is a positive definite function. Then the following conjecture looks quite reasonable. 

Conjecture 7.8. The matrix S„^fe is positive definite; in particular, one has detS„_fe > 0. In other words, 
{DT{X)}TeSTah{{k")) g^ves another basis of the space ^Jfe"'*'* = C[6fe„]-wrdetfe. 



K. Kimoto and M. Wakayama. 
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We try to examine the first few examples which may support the above conjecture. 
Example 7.9. We have 

, „ 265 /3\" , „ 3 /5^" 

det^4,2 = ^(^gj ' '^^*^^'^=2^U 

We notice here that 



f(^') = 2, = /(3^) = 5, = = 14. 



A Appendix : Laplace expansion of a-determinants 

Proposition A.l (Laplace expansion). For a given n by n matrix X = (a;jj)i<ij<„, we have 



p=i 



where Xpq is a n — I by n — 1 matrix obtained by the following procedure: (1) remove q-th column vector and 
q-th row vector in X, (2) if p ^ q, then replace the row vector {xpi, . . . ,Xpn) in X by {Xgi, . . . ,Xqn)- 



Proof. We have 



■g{i)i 



P=i ffes„ 

n 



n-iy„((p,g)-g) 



n 



X 



p=l 


ses„ 


l<»(7^9)<n 


n 




-l)-!^„_i(a) TT 

11 ■^{p,q)-g{vi' 


p=i 


g(q)=q 




n 


^'^iz^pgdet^ ^ Xpq 




p=i 







Here we use the fact that i^n{{p, q) ■ 9) = i^n-iid) + ^pq if 5(9) = 9 (see the proof of Lemma 2.1). 

(xii X\2 Xl3 Xl4\ 

X2I X22 X2-i X2A 
X31 X32 X33 X34 

\x4,i X42 3^43 a;44y 



Example A.2 (n = 4). For X 



we have 



X\2 — 



X: 



32 



'X21 X23 a;24 

2:31 a;33 a;34 

\^Xil XiS Xn J 

'xxx xxz Xxa" 

X21 X23 X24 

\X4i X43 Xii I 



X22 — 



X. 



42 



Xll 


Xl3 


Xii 


X31 


X33 


X34 


Xii 


Xi3 


Xii 


Xll 


Xl3 


Xii 


X31 


X33 


X3i 


X21 


X23 


X24 



□ 
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Hence we have 

fxii 
X21 



det(") 



X31 



X12 
X22 
X32 
X42 



Xl3 Xi4\^ 

X23 X24, 

X33 X34 

X43 XaJ 



=axi2 



det(") 



01x32 



de&'> 



X22 




det(") 



ax42 det 
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